Method for scheduling user request in distributed resource system, and apparatus

ABSTRACT

According to a method for scheduling a user request in a distributed resource system, an apparatus, and a system that are provided by embodiments of the present invention, in a T n+1  period, an S d  acquires, from a coordinator G k  of a user z, a resource C z (T n ) that is consumed by a user z request in a T n  period, and the S d  schedules, according to φ z , C z (T n ), C z,d (T n ), and N z,d (T n ), a P i   z,d  by using a scheduling algorithm. The user z request can be scheduled without depending on a user agent. In addition, the S d  schedules, according to φ z , C z (T n ), C z,d (T n ), and N z,d (T n ), the P i   z,d  by using the scheduling algorithm, thereby implementing global scheduling on the user z request and ensuring a performance requirement of the user z.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 14/932,628, filed on Nov. 4, 2015, now allowed, which is a continuation of International Application No. PCT/CN2014/095381,filed on Dec. 29, 2014, both of which are hereby incorporated by reference in their entireties.

TECHNICAL FIELD

The present invention relates to the information technology field, and in particular, to a method for scheduling a user request in a distributed resource system, and an apparatus.

BACKGROUND

There are multiple resource-providing entities in a distributed resource system. Resources include a computing capability of a Central Processing Unit (CPU), hard disk space, memory space, network bandwidth, and the like, where a resource may be a physical resource or a virtual resource. When a resource-providing entity provides a resource according to a user request, a resource competition occurs between users. Therefore, scheduling a user request to ensure performance for users becomes extremely important.

FIG. 1 shows a user request scheduling solution, where a distributed resource system shown in FIG. 1 is a distributed file system and includes a user agent A, a user agent B, a scheduler A, a scheduler B, a resource-providing entity A, and a resource-providing entity B, where the user agent A does not communicate with the user agent B. The user agent A queries a metadata server in the distributed file system according to a user A request, and the user agent A determines, according to information obtained by querying, a scheduler in FIG. 1 to which the user A request is sent. Likewise, the user agent B queries the metadata server in the distributed file system according to a user B request, and the user agent B determines, according to information obtained by querying, a scheduler in FIG. 1 to which the user B request is sent. The resource-providing entity A and the resource-providing entity B are file systems. The resource-providing entity A is configured to provide a resource A, where the resource A is specifically a capability of providing Input/Output Operations Per Second (IOPS). The resource-providing entity B is configured to provide a resource B, where the resource B is specifically a capability of providing IOPS. A user A and a user B each send a request to the distributed resource system. For example, the user A sends a user A request to each of the resource-providing entity A and the resource-providing entity B by using the user agent A; then, the resource-providing entity A provides the resource A for the user A request, and the resource-providing entity B provides the resource B for the user A request. The user B sends a user B request to each of the resource-providing entity A and the resource-providing entity B by using the user agent B, and the resource-providing entity A and the resource-providing entity B provide the resources for the user B request. When the user A sends the user A request to each of the resource-providing entity A and the resource-providing entity B, in an implementation manner in FIG. 1, the resources provided by the resource-providing entity A and the resource-providing entity B for the user A are both IOPS capabilities. Likewise, the resources provided by the resource-providing entity A and the resource-providing entity B for the user B are also IOPS capabilities.

In the distributed resource system shown in FIG. 1, a resource weight is allocated to each of the user A and the user B. For example, a resource weight of the user A is 2, and a resource weight of the user B is 1. A resource weight of a user indicates a resource quota that is allocated by a distributed resource system to the user. The user agent A collects statistics on a quantity of user A requests that are sent by the user A to the scheduler A and the scheduler B, and the user agent B collects statistics on a quantity of user B requests that are sent by the user B to the scheduler A and the scheduler B. The scheduler A allocates, from the resource-providing entity A, a resource for the user A request according to a resource weight of the user A and the quantity of the user A requests that are sent by the user agent A to the scheduler A and the scheduler B. The scheduler A allocates, from the resource-providing entity A, a resource for the user B request according to a resource weight of the user B and the quantity of the user B requests that are sent by the user agent B to the scheduler A and the scheduler B. Likewise, the scheduler B allocates, from the resource-providing entity B, a resource for the user A request according to the resource weight of the user A and the quantity of the user A requests that are sent by the user agent A to the scheduler A and the scheduler B. The scheduler B allocates, from the resource-providing entity B, a resource for the user B request according to the resource weight of the user B and the quantity of the user B requests that are sent by the user agent B to the scheduler A and the scheduler B.

Because the user agent A and the user agent B are applicable only to a distributed file system scenario, the solution in an architecture shown in FIG. 1 cannot be widely applied to a scenario in which a user request is scheduled in a distributed resource system.

SUMMARY

According to a first aspect, an embodiment of the present invention provides a method for scheduling a user request in a distributed resource system, where the distributed resource system includes schedulers S_(x), resource-providing entities R_(x), and a coordinator G_(y), where x is any one of consecutive natural numbers from 1 to M, and M≧2; y is any one of consecutive natural numbers from 1 to Y, and Y is a natural number; the S_(x) communicates with the R_(x); the G_(y) communicates with any S_(x); and the method includes:

acquiring, by an S_(d) in a T_(n+1) period and from a coordinator G_(k) of a user z, a resource C_(z)(T_(n)) that is consumed by a user z request in a T_(n) period, where

${{C_{z}\left( T_{n} \right)} = {\sum\limits_{x = 1}^{M}\; {C_{z,x}\left( T_{n} \right)}}};$

d and k are natural numbers, 1≦d≦M, and 1≦k≦Y; a resource weight of the user z is φ_(z); C_(z,x)(T_(n)) is a quantity of resources that are provided by the R_(x) and consumed by N_(z,x)(T_(n)) user z requests received by the S_(x) in the T_(n) period; and z indicates an identifier of the user; and

scheduling, by the S_(d) according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N _(z,d)(T_(n)), a P^(i) _(z,d) by using a scheduling algorithm, where the P^(i) _(z,d) is the i^(th) user z request received by the S_(d), and C_(z,d)(T_(n)) is a quantity of resources that are provided by an R_(d) and consumed by N_(z,d)(T_(n)) user z requests received by the S_(d) in the T_(n) period.

With reference to the first aspect of the embodiment of the present invention, in a first possible implementation manner, the scheduling, by the S_(d) according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)), a P^(i) _(z,d) by using a scheduling algorithm specifically includes:

computing, by the S_(d) according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)), a virtual start time S(P^(i) _(z,d)) and a virtual finish time F(P^(i) _(z,d)) of the P^(i) _(z,d); and adding the P^(i) _(z,d) to a scheduling queue, where the scheduling queue ranks the user request according to a value of the virtual start time of the user request.

With reference to the first possible implementation manner of the first aspect of the embodiment of the present invention, in a second possible implementation manner, S(P^(i) _(z,d))=max{v(P^(i) _(z,d)), F(P^(i−1) _(z,d))},

${{F\left( P_{z,d}^{i} \right)} = {{S\left( P_{z,d}^{i} \right)} + \frac{c\left( P_{z,d}^{i} \right)}{\varphi_{z}} + \frac{d\left( {P_{z,d}\left( T_{n + 1} \right)} \right)}{\varphi_{z}}}},{{{and}\mspace{14mu} {d\left( {P_{z,d}\left( T_{n + 1} \right)} \right)}} = \frac{{C_{z}\left( T_{n} \right)} - {C_{z,d}\left( T_{n} \right)}}{N_{z,d}\left( T_{n} \right)}},$

where v(P^(i) _(z,d)) indicates a virtual time of the S_(d) when the S_(d) receives the P^(i) _(z,d), and c(P^(i) _(z,d)) indicates a resource that is provided by the R_(d) and consumed by the P^(i) _(z,d).

With reference to the first possible implementation manner of the first aspect of the embodiment of the present invention, in a third possible implementation manner, S(P^(i) _(z,d))=max{v(P^(i) _(z,d)),

$\left. {{F\left( P_{z,d}^{i - 1} \right)} + \frac{d\left( {P_{z,d}\left( T_{n + 1} \right)} \right)}{\varphi_{z}}} \right\},{{F\left( P_{z,d}^{i} \right)} = {{S\left( P_{z,d}^{i} \right)} + \frac{c\left( P_{z,d}^{i} \right)}{\varphi_{z}}}},{{{and}\mspace{14mu} {d\left( {P_{z,d}\left( T_{n + 1} \right)} \right)}} = \frac{{C_{z}\left( T_{n} \right)} - {C_{z,d}\left( T_{n} \right)}}{N_{z,d}\left( T_{n} \right)}},$

where v(P^(i) _(z,d)) indicates a virtual time of the S_(d) when the S_(d) receives the P^(i) _(z,d), and c(P^(i) _(z,d)) indicates a resource that is provided by the R_(d) and consumed by the P^(i) _(z,d).

According to a second aspect, an embodiment of the present invention provides a scheduler S_(d), where the scheduler S_(d) is applied to a distributed resource system, and the distributed resource system includes schedulers S_(x), resource-providing entities R_(x), and a coordinator G_(y), where x is any one of consecutive natural numbers from 1 to M, and M≧2; d is a number in x; y is any one of consecutive natural numbers from 1 to Y; the S_(x) communicates with the R_(x); the G_(y) communicates with any S_(x); the scheduler S_(d) includes a central processing unit and a memory, where the central processing unit executes an executable instruction in the memory, to perform the method in any one of the first aspect to the third possible implementation manner of the first aspect of the embodiment of the present invention.

According to a third aspect, an embodiment of the present invention provides a scheduler S_(d), where the scheduler S_(d) is applied to a distributed resource system, and the distributed resource system includes schedulers S_(x), resource-providing entities R_(x) and a coordinator G_(y), where x is any one of consecutive natural numbers from 1 to M, and M≧2; d is a number in x; y is any one of consecutive natural numbers from 1 to Y; the S_(x) communicates with the R_(x); the G_(y) communicates with any S_(x); and the scheduler S_(d) includes an acquiring unit and a scheduling unit, where:

the acquiring unit is configured to acquire, in a T_(n+1) period and from a coordinator G_(k) of a user z, a resource C_(z)(T_(n)) that is consumed by a user z request in a T_(n) period, where

${{C_{z}\left( T_{n} \right)} = {\sum\limits_{x = 1}^{M}\; {C_{z,x}\left( T_{n} \right)}}};$

k is a natural number, and 1≦k≦Y; a resource weight of the user z is φ_(z); C_(z,x)(T_(n)) is a quantity of resources that are provided by the R_(x) and consumed by N_(z,x)(T_(n)) user z requests received by the S_(x) in the T_(n) period; and z indicates an identifier of the user; and

the scheduling unit is configured to schedule, according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)), P^(i) _(z,d) by using a scheduling algorithm, where P^(i) _(z,d) is the i^(th) user z request received by the S_(d), and C_(z,d)(T_(n)) is resource that is provided by an R_(d) and consumed by N_(z,d)(T_(n)) user z requests received by the S_(d) in the T_(n) period.

According to a fourth aspect, an embodiment of the present invention provides a non-volatile computer readable storage medium, where the non-volatile computer readable storage medium stores a computer instruction that is used to perform user request scheduling in a distributed resource system, and the distributed resource system includes schedulers S_(x), resource-providing entities R_(x), and a coordinator G_(y), where x is any one of consecutive natural numbers from 1 to M, and M≧2; y is any one of consecutive natural numbers from 1 to Y, and Y is a natural number; the S_(x) communicates with the R_(x), and the G_(y) communicates with any S_(x); and a scheduler S_(d) executes the computer instruction, to perform the method in any one of the first aspect to the third possible implementation manner of the first aspect of the embodiment of the present invention, where d is a number in x.

According to a fifth aspect, an embodiment of the present invention provides a distributed resource system, where the distributed resource system includes schedulers S_(x), resource-providing entities R_(x), and a coordinator G_(y), where x is any one of consecutive natural numbers from 1 to M, and M≧2; y is any one of consecutive natural numbers from 1 to Y, and Y is a natural number; the S_(x) communicates with the R_(x); and the G_(y) communicates with any S_(x);

a coordinator G_(k) of a user z is configured to provide, for a scheduler S_(d) in a T_(n+1) period, a resource C_(z)(T_(n)) that is consumed by a user z request in a T_(n) period, where

${{C_{z}\left( T_{n} \right)} = {\sum\limits_{x = 1}^{M}\; {C_{z,x}\left( T_{n} \right)}}};$

d and k are natural numbers, 1≦d≦M, and 1≦k≦Y; a resource weight of the user z is φ_(z); C_(z,x)(T_(n)) is a quantity of resources that are provided by the R_(x) and consumed by N_(z,x)(T_(n)) user z requests received by the S_(x) in the T_(n) period; and z indicates an identifier of the user; and

the scheduler S_(d) is configured to schedule, according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)), a P^(i) _(z,d) by using a scheduling algorithm, where the P^(i) _(z,d) is the i^(th) user z request received by the S_(d), and C_(z,d)(T_(n))is a quantity of resources that are provided by an R_(d) and consumed by N_(z,d)(T_(n)) user z requests received by the S_(d) in the T_(n) period.

According to a sixth aspect, an embodiment of the present invention provides a resource scheduling method in a distributed resource system, where the distributed resource system includes multiple schedulers;

a first scheduler in the multiple schedulers acquires, from a coordinator of the first user, the sum of resources that are consumed in the multiple schedulers by a user request of the first user in a previous period; and

the first scheduler schedules the user request of the first user according to a resource weight of the first user, the sum of resources that are consumed in the multiple schedulers by the user request of the first user in the previous period, a resource that is consumed in the first scheduler by the user request of the first user in the previous period, and a quantity of user requests of the first user received by the first scheduler in the previous period.

With reference to the sixth aspect, in a first possible implementation manner, that the first scheduler schedules the user request of the first user according to a resource weight of the first user, the sum of resources that are consumed in the multiple schedulers by the user request of the first user in the previous period, a resource that is consumed in the first scheduler by the user request of the first user in the previous period, and a quantity of user requests of the first user received by the first scheduler in the previous period specifically includes:

computing, by the first scheduler according to the resource weight of the first user, the sum of resources that are consumed in the multiple schedulers by the user request of the first user in the previous period, the resource that is consumed in the first scheduler by the user request of the first user in the previous period, and the quantity of the user requests of the first user received by the first scheduler in the previous period, a virtual start time and a virtual finish time of the user request of the first user; and adding the user request of the first user to a scheduling queue, where the scheduling queue ranks the user request of the first user according to a value of the virtual start time of the user request.

According to a seventh aspect, an embodiment of the present invention provides a distributed resource system, where the distributed resource system includes multiple schedulers, and a first scheduler in the multiple schedulers includes an acquiring unit and a scheduling unit, where:

the acquiring unit is configured to acquire, from a coordinator of the first user, the sum of resources that are consumed in the multiple schedulers by a user request of the first user in a previous period; and

the scheduling unit is configured to schedule the user request of the first user according to a resource weight of the first user, the sum of resources that are consumed in the multiple schedulers by the user request of the first user in the previous period, a resource that is consumed in the first scheduler by the user request of the first user in the previous period, and a quantity of user requests of the first user received by the first scheduler in the previous period.

With reference to the seventh aspect, in a first possible implementation manner, the scheduling unit is specifically configured to compute, according to the resource weight of the first user, the sum of resources that are consumed in the multiple schedulers by the user request of the first user in the previous period, the resource that is consumed in the first scheduler by the user request of the first user in the previous period, and the quantity of the user requests of the first user received by the first scheduler in the previous period, a virtual start time and a virtual finish time of the user request of the first user; and add the user request of the first user to a scheduling queue, where the scheduling queue ranks the user request of the first user according to a value of the virtual start time of the user request.

According to the method for scheduling a user request in a distributed resource system, the apparatus, and the system that are provided by embodiments of the present invention, in a T_(n+1) period, an S_(d) acquires, from a coordinator G_(k) of a user z, a resource C_(z)(T_(n)) that is consumed by a user z request in a T_(n) period, and the S_(d) schedules, according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)), a P^(i) _(z,d) by using a scheduling algorithm. The user z request can be scheduled without depending on a user agent. In addition, the S_(d) schedules, according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)), the P^(i) _(z,d) by using the scheduling algorithm, thereby implementing global scheduling on the user z request and ensuring a performance requirement of the user z.

BRIEF DESCRIPTION OF DRAWINGS

To describe the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings required for describing the embodiments.

FIG. 1 is a schematic diagram of a distributed resource system in the prior art;

FIG. 2 is a schematic diagram of a distributed resource system according to an embodiment of the present invention;

FIG. 3A is a diagram of a relationship among a scheduler A, a resource-providing entity A, and a coordinator A;

FIG. 3B is a diagram of a relationship among a scheduler A, a resource-providing entity A, and a coordinator A;

FIG. 4 is a schematic diagram of a method according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a status of a distributed resource system according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of a status of user requests in a scheduling queue;

FIG. 7 is a schematic diagram of a status of user requests in a scheduling queue;

FIG. 8 is a schematic diagram of a status of a distributed resource system according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of a status of user requests in a scheduling queue;

FIG. 10 is a schematic diagram of a status of user requests in a scheduling queue;

FIG. 11 is an architecture diagram of a distributed resource system according to an embodiment of the present invention; and

FIG. 12 is a schematic structural diagram of a scheduler according to an embodiment of the present invention.

DESCRIPTION OF EMBODIMENTS

The following clearly describes the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention.

As shown in FIG. 2, an embodiment of the present invention provides a distributed resource system. The distributed resource system includes a scheduler A, a scheduler B, a resource-providing entity A, a resource-providing entity B, a coordinator A, and a coordinator B. The scheduler A communicates with the resource-providing entity A, and is configured to schedule a user request sent to the resource-providing entity A. The scheduler B communicates with the resource-providing entity B, and is configured to schedule a user request sent to the resource-providing entity B. The coordinator A communicates with the scheduler A and the scheduler B, and the coordinator B communicates with the scheduler A and the scheduler B. A resource in this embodiment of the present invention may be a physical resource or a virtual resource, such as a CPU computing capability, hard disk space, a memory space, network bandwidth, or the like. A user in this embodiment of the present invention may refer to a client, a virtual machine, an application process, or the like, which is not specifically limited in this embodiment of the present invention. A resource allocated by a resource-providing entity for a user A request and a resource allocated by the resource-providing entity for a user B request are resources of a same type. Resources that are provided by the resource-providing entity A and the resource-providing entity B for a user A request and resources that are provided by the resource-providing entity A and the resource-providing entity B for a user B request are resources of a same type. In this embodiment of the present invention, a resource allocated by a resource-providing entity to a user request has the same meaning as a resource provided by a resource-providing entity for a user request, which may also be expressed as a resource of a resource-providing entity that is consumed by a user request, or a resource that is provided by a resource-providing entity and consumed by a user request. A resource that is provided by a resource-providing entity and consumed by a user request refers to a resource that needs to be consumed by the user request, which specifically includes a resource that has been consumed, or includes, if the user request has already been processed by the resource-providing entity, a resource that needs to be consumed by the user request after the user request is received by a scheduler and before the user request is processed by the resource-providing entity. In an implementation manner, after receiving a user request, a scheduler can calculate a quantity of resources that are provided by a resource-providing entity and need to be consumed by the user request. For example, if a scheduler A receives 100 user requests in a period, resources that are provided by a resource-providing entity and consumed by the 100 user requests can be calculated by summing up resources that are provided by the resource-providing entities and need to be consumed by all user requests.

This embodiment of the present invention is described by using an example in which a resource is an IOPS capability, the resource-providing entity A is a storage array A, and the resource-providing entity B is a storage array B.

Before this embodiment of the present invention is further described, it should be noted that, a relationship among the scheduler A, the coordinator A, and the resource-providing entity A shown in FIG. 2 is just a logical representation. In specific implementation, the resource-providing entity A, the scheduler A, and the coordinator A may be located in a same apparatus, as shown in FIG. 3a ; or, the resource-providing entity A and the scheduler A are located in a same apparatus while the coordinator A is located independently of the resource-providing entity A and the scheduler A, as shown in FIG. 3b ; or, another implementation manner may be used. A specific implementation manner is not limited herein. Likewise, for a relationship among the scheduler B, the coordinator B, and the resource-providing entity B shown in FIG. 2, refer to FIG. 3a and FIG. 3b . The scheduler may be an independent device or a hardware module. A function of the scheduler may be further implemented by a processor by executing a specific computer instruction. For example, if functions of the scheduler A, the coordinator A, and the resource-providing entity A are all implemented by a processor by executing specific computer instructions, communication between the scheduler A and the coordinator A, and communication between the scheduler A and the resource-providing entity A may be presented as instruction invoking or links between different program modules. Likewise, the coordinator may be an independent device or a hardware module. A function of the coordinator may be further implemented by a processor by executing a specific computer instruction, which is not limited in this embodiment of the present invention. In specific implementation, steps of the method implemented by a scheduler in this embodiment of the present invention may be implemented by a computer processor by executing a computer instruction in a memory. Details are not described herein again.

In the distributed resource system shown in FIG. 2, when a user A sends a user A request to the storage array A and the storage array B, both the storage array A and the storage array B need to consume an IOPS capability when processing the user A request. Likewise, when a user B sends a user B request to the storage array A and the storage array B, IOPS capabilities of the storage array A and the storage array B also need to be consumed. Particularly, when the IOPS capability of the storage array A cannot meet a requirement of processing the user A request and the user B request at the same time, the scheduler A needs to schedule the received user A request and the received user B request according to a scheduling algorithm; when the IOPS capability of the storage array B cannot meet a requirement of processing the user A request and the user B request at the same time, the scheduler B needs to schedule the received user A request and the received user B request according to a scheduling algorithm.

There is a resource competition relationship between the user A and the user B. To ensure performance for the users, a resource weight is generally allocated to each user in the distributed resource system. The user A and the user B are used as an example in this embodiment of the present invention, where a resource weight φ_(A) of the user A is 2, and a resource weight φ_(B) of the user B is 1. Then a ratio of the resource weight of the user A to the resource weight of the user B is 2:1. The resource weights of the user A and the user B are configured in both the scheduler A and the scheduler B. Specifically, the resource weight of the user A may be delivered by the user A to each of the scheduler A and the scheduler B, and the resource weight of the user B is delivered by the user B to each of the scheduler A and the scheduler B; the scheduler A and the scheduler B perform the foregoing configuration. A form of a resource weight is not limited to the form described in this embodiment of the present invention, and may also be that the resource weight φ_(A) of the user A is ⅔, and the resource weight φ_(B) of the user B is ⅓.

The scheduler A receives the user A request sent by the user A, and the scheduler A receives the user B request sent by the user B. The storage array A needs to provide the IOPS capability for both the user A request and the user B request, or in other words, both the user A request and the user B request consume the IOPS capability provided by the storage array A. Likewise, the scheduler B receives the user A request sent by the user A, and the scheduler B receives the user B request sent by the user B. When the IOPS capability of the storage array A can meet the user A request and the user B request, the scheduler A does not need to schedule the received user A request and the received user B request according to a scheduling algorithm; when the IOPS capability of the storage array B can meet the user A request and the user B request, the scheduler B does not need to schedule the received user A request and the received user B request according to a scheduling algorithm. The scheduler A only needs to compute virtual start times and virtual finish times of the received user requests according to a scheduling algorithm. Likewise, the scheduler B also only needs to compute virtual start times and virtual finish times of the received user requests according to a scheduling algorithm. For meanings of a virtual start time and a virtual finish time, refer to a Start-time Fairness Queuing (SFQ) algorithm. When the user requests need to be scheduled, the scheduler A needs to add, according to a scheduling algorithm, the received user A request and the received user B request to a scheduling queue of the scheduler A for scheduling; the scheduler B also needs to add, according to the scheduling algorithm, the received user A request and the received user B request to a scheduling queue of the scheduler B for scheduling.

As shown in FIG. 4, in step 401, a scheduler A acquires, in a T_(n+1) period and from a home coordinator A of a user A, a resource C_(A)(T_(n)) that is consumed by a user A request in a T_(n) period, and acquires, from a home coordinator B of a user B, a resource C_(B)(T_(n)) that is consumed by a user B request in the T_(n) period, where T_(n) indicates the n^(th) period, and T_(n+1) indicates the (n+1)^(th) period; C_(A)(T_(n)) indicates the sum of a resource C_(A,A)(T_(n)) that is provided by a resource-providing entity A and consumed by N_(A,A)(T_(n)) user A requests received by the scheduler A in the T_(n) period, and a resource C_(A,B)(T_(n)) that is provided by a resource-providing entity B and consumed by N_(A,B)(T_(n)) user A requests received by a scheduler B in the T_(n) period; and C_(B)(T_(n)) indicates the sum of a resource C_(B,A)(T_(n)) that is provided by the resource-providing entity A and consumed by N_(B,A)(T_(n)) user B requests received by the scheduler A in the T_(n) period, and a resource C_(B,B)(T_(n)) that is provided by the resource-providing entity B and consumed by N_(B,B)(T_(n)) user B requests received by the scheduler B in the T_(n) period. The home coordinator A of the user A refers to a coordinator that acquires, in a distributed resource system, the sum of a resource of the storage array A that is consumed by the user A request processed by the scheduler A in the T_(n) period, and a resource of the storage array B that is consumed by the user A request processed by the scheduler B in the T_(n) period. The home coordinator A of the user A is also called a coordinator A of the user A. Likewise, a home coordinator of a user is also called a coordinator of the user.

The scheduler A acquires, in the T_(n) period and according to a resource that is provided by the resource-providing entity A and consumed by each received user A request, the resource C_(A,A)(T_(n)) that is provided by the resource-providing entity A and consumed by the N_(A,A)(T_(n)) user A requests processed by the scheduler A in the T_(n) period; and the scheduler A acquires, in the T_(n) period and according to a resource that is provided by the resource-providing entity A and consumed by each received user B request, the resource C_(B,A)(T_(n)) that is provided by the resource-providing entity A and consumed by the N_(B,A)(T_(n)) user B requests processed by the scheduler A in the T_(n) period. Likewise, the scheduler B acquires, in the T_(n) period and according to a resource that is provided by the resource-providing entity B and consumed by each received user A request, the resource C_(A,B)(T_(n)) that is provided by the resource-providing entity B and consumed by the N_(A,B) (T_(n)) user A requests processed by the scheduler B in the T_(n) period; and the scheduler B acquires, in the T_(n) period and according to a resource that is provided by the resource-providing entity B and consumed by each received user B request, the resource C_(B,B)(T_(n)) that is provided by the resource-providing entity B and consumed by the N_(B,B)(T_(n)) user B requests processed by the scheduler B in the T_(n) period. Resources consumed by the user A requests processed in the T_(n) period are C_(A)(T_(n))=C_(A,A)(T_(n))+C_(A,B)(T_(n)), and resources consumed by the user B requests processed in the T_(n) period are C_(B)(T_(n))=C_(B,A)(T_(n))+C_(B,B)(T_(n)).

Specifically, the coordinator A acquires C_(A,A)(T_(n)) and C_(A,B)(T_(n)). An implementation manner is that the scheduler A proactively sends the C_(A,A)(T_(n)) to the coordinator A, and the scheduler B proactively sends the C_(A,B)(T_(n)) to the coordinator A. Another implementation manner is that the coordinator A requests to acquire the C_(A,A)(T_(n)) from the scheduler A, and the coordinator A requests to acquire the C_(A,B)(T_(n)) from the scheduler B. Specifically, the coordinator A may acquire the C_(A,A)(T_(n)) and the C_(A,B)(T_(n)) at one time, or the coordinator A may communicate with the scheduler A and the scheduler B in real time, to acquire, in real time, the resource that is provided by the resource-providing entity A and consumed by each user A request received by the scheduler A, and acquire, in real time, the resource that is provided by the resource-providing entity B and consumed by each user A request received by the scheduler B. For a manner in which the coordinator B acquires the resources C_(B)(T_(n)) consumed for processing the user B requests, refer to the manner of the coordinator A, and details are not described herein again.

The scheduler A acquires, from the home coordinator A of the user A, the resource C_(A)(T_(n)) consumed by the user A requests in the T_(n) period, and acquires, from the home coordinator B of the user B, the resource C_(B)(T_(n)) consumed by the user B requests in the T_(n) period. In an implementation manner, in a T_(n+1) period, the coordinator A sends the C_(A) (T_(n)) to each of the scheduler A and the scheduler B, and the coordinator B sends the C_(B)(T_(n)) to each of the scheduler A and the scheduler B. In another implementation manner, in a T_(n+1) period, the scheduler A requests the C_(A) (T_(n)) from the coordinator A, the scheduler A requests the C_(B)(T_(n)) from the coordinator B; and the scheduler B requests the C_(A)(T_(n)) from the coordinator A, the scheduler B requests the C_(B)(T_(n)) from the coordinator B. The foregoing two manners are both described as: in a T_(n+1) period, the scheduler A acquires, from the home coordinator A of the user A, the resources C_(A)(T_(n)) consumed by the user A requests in the T_(n) period, and acquires, from the home coordinator B of the user B, the resources C_(B)(T_(n)) consumed by the user B requests in the T_(n) period. It can be known from the foregoing description that, in either of the foregoing implementation manners, the scheduler A stores the N_(A,A)(T_(n)), N_(B,A)(T_(n)), C_(A,A)(T_(n)), and C_(B,A)(T_(n)); and the scheduler B stores the N_(A,B)(T_(n)), N_(B,B)(T_(n)), C_(A,B)(T_(n)), and C_(B,B)(T_(n)).

As for determining of a home coordinator of a user, a manner may be that a coordinator is configured as a home coordinator of a user; or when an identifier of a user is an integer, a modulo operation is performed on a total quantity of coordinators according to the identifier of the user, to determine a home coordinator of the user; or a hash function may be used to perform computation on an identifier of a user to obtain a hash value, and then a modulo operation is performed on a total quantity of coordinators by using the hash value, to determine a home coordinator of the user. For example, there are 100 users and 20 coordinators, and identifiers of the users are from 1 to 100, the 20 coordinators are numbered from 1 to 20, and a modulo operation is performed on a total quantity of the coordinators according to the identifiers of the users, to determine a home coordinator of each user. The present invention sets no limitation thereto. A total quantity of coordinators may be less than or equal to a quantity of schedulers, and each coordinator can communicate with any scheduler.

Step 402. The scheduler A schedules, according to φ_(A), C_(A)(T_(n)), C_(A,A)(T_(n)), and N_(A,A)(T_(n)), the received user A request by using a scheduling algorithm; and the scheduler A schedules, according to φ_(B), C_(B)(T_(n)), C_(B,A)(T_(n)), and N_(B,A)(T_(n)), the received user B request by using the scheduling algorithm.

Likewise, the scheduler B schedules, according to φ_(A), C_(A)(T_(n)), C_(A,B)(T_(n)), and N_(A,B)(T_(n)), a received user A request by using a scheduling algorithm; and the scheduler B schedules, according to φ_(B), C_(B)(T_(n)), C_(B,B)(T_(n)), and N_(B,B)(T_(n)), a received user B request by using the scheduling algorithm.

When a user request in a scheduler is scheduled according to the method shown in FIG. 4, the scheduler A performs scheduling according to a quantity of resources that are consumed by the user A request in the distributed system in a previous period, a resource weight of the user A, a quantity of user A requests received by the scheduler A in the previous period, and the resource that is provided by the resource-providing entity A and consumed by the user A request received by the scheduler A in the previous period, where the quantity of resourcesthat are consumed by the user A request in the distributed system in a previous period is acquired from the coordinator A. In an architecture without a user agent in the prior art, a coordinator is used to acquire a quantity of resources that are consumed by a user A request in a distributed system in a previous period, so that a scheduler A does not need to perform an extra operation, which can reduce a resource usage of the scheduler A. In addition, the scheduler schedules the user A request according to φ_(A), C_(A)(T_(n)), C_(A,A)(T_(n)), and N_(A,A)(T_(n)), thereby implementing global scheduling on the user A requests and ensuring a performance requirement of a user z.

When the scheduler A schedules the user B request, or the scheduler B schedules the user A request, or the scheduler B schedules the user B request, the same foregoing effect can also be achieved. In a user agent-based architecture in the prior art, a scheduler can acquire only a user request and statistics of the user request that are sent by one user agent; therefore, it is difficult to implement global scheduling on a same user request.

In this embodiment of the present invention, that a scheduler A schedules a user A request is used as an example. During global scheduling on the user A request, the scheduler A needs to consider C_(A)(T_(n)) in addition to φ_(A), C_(A,A)(T_(n)), and N_(A,A)(T_(n)) in a previous period, thereby implementing scheduling of the user A request and further ensuring performance for a user A.

Specifically, in this embodiment of the present invention, P^(i) _(A,A) indicates the i^(th) user A request that is sent by the user A and received by the scheduler A, where a storage array A provides a resource for the P^(i) _(A,A); and P^(i) _(A,B) indicates the i^(th) user A request that is sent by the user A and received by a scheduler B, where a storage array B provides a resource for the P^(i) _(A,B). P^(k) _(B,A) indicates the k^(th) user B request that is sent by a user B and received by the scheduler A, where the storage array A provides a resource for the P^(k) _(B,A); and P^(k) _(B,B) indicates the k^(th) user B request that is sent by the user B and received by the scheduler B, where the storage array B provides a resource for the P^(k) _(B,B). In this embodiment of the present invention, φ_(A)=2, and φ_(B)=1.

In a T_(n) period, when a quantity of user A requests and user B requests that are received by the scheduler A is less than an IOPS capability of the storage array A, there is no need to schedule, by using a scheduling algorithm, the user A requests and the user B requests that are received by the scheduler A, and the user A requests and the user B requests are directly processed by the storage array A. When a quantity of user A requests and user B requests that are received by the scheduler B is less than an IOPS capability of the storage array B, there is also no need to schedule, by using a scheduling algorithm, the user A requests and the user B requests that are received by the scheduler B, and the user A requests and the user B requests are directly processed by the storage array B. However, a home coordinator A of the user A needs to acquire resources that are consumed by user A requests received by schedulers in the T_(n) period. In this embodiment of the present invention, the home coordinator A of the user A acquires a resource C_(A)(T_(n))that is consumed by the user A requests that are received by the scheduler A and the scheduler B in the T_(n) period. A home coordinator B of the user B also acquires a resource C_(B)(T_(n))that is consumed by the user B requests that are received by the scheduler A and the scheduler B in the T_(n) period. The coordinator A sends C_(A)(T_(n)) to the scheduler A and the scheduler B, and the coordinator B sends C_(B)(T_(n))to the scheduler A and the scheduler B. When a quantity of the user A requests received by the scheduler A in the T_(n) period is N_(A,A)(T_(n)), a consumed resource that is provided by the storage array A is C_(A,A)(T_(n)); when a quantity of the user B requests received by the scheduler A in the T_(n) period is N_(B,A)(T_(n)) a consumed resource that is provided by the storage array A is C_(B,A)(T_(n)). When a quantity of the user A requests received by the scheduler B in the T_(n) period is N_(A,B)(T_(n)), a consumed resource that is provided by the storage array B is C_(A,B)(T_(n)); when a quantity of the user B requests received by the scheduler B in the T_(n) period is N_(B,B)(T_(n)), a consumed resource that is provided by the storage array A is C_(B,B)(T_(n)).

The scheduler A is used as an example. In a T_(n+1) period, the scheduler A acquires, from the home coordinator A of the user A, a resource C_(A)(T_(n)) that is consumed by a user A request in the T_(n) period. In the T_(n+1) period, when an IOPS capability of the storage array A cannot meet a user A request and a user B request, the scheduler A adds, by using a scheduling algorithm, the received user A request and the received user B request to a scheduling queue of the scheduler A for scheduling. When receiving the user A request and the user B request, the scheduler A computes a virtual start time and a virtual finish time of the user A request, and a virtual start time and a virtual finish time of the user B request. The user A request and the user B request are added to the scheduling queue, and the user A request and the user B request are ranked in the scheduling queue in ascending order of the virtual start times for processing. A user request whose virtual start time is the earliest is ranked at the queue head of the scheduling queue, and a user request whose virtual start time is the latest is ranked at the queue tail of the scheduling queue. The queue head of the scheduling queue refers to a position at which the storage array A performs user request processing first in the scheduling queue. When processing the user requests, the storage array A acquires a user request from the queue head of the scheduling queue for processing, that is, a user request whose virtual start time is the earliest is processed first. The storage array A processes the user requests in the scheduling queue in ascending order of the virtual start times of the user requests.

In a first implementation manner, a virtual start time of a P^(i) _(A,A) that is received by the scheduler A is represented by S(P^(i) _(A,A)), and a virtual finish time of the P^(i) _(A,A) is represented by F(P^(i) _(A,A)). S(P^(i) _(A,A))=max{v(P^(i) _(A,A)), F(P^(i−1) _(A,A))}, where v(P^(i) _(A,A)) indicates a virtual time of the scheduler A when the scheduler A receives the P^(i) _(A,A), F(P^(i−1) _(A,A)) indicates a virtual finish time of a P^(i−1) _(A,A), and max{v(P^(i) _(A,A)), F(P^(i−1) _(A,A))} indicates a maximum value in v(P^(i) _(A,A)) and F(P^(i−1) _(A,A)).

${F\left( P_{A,A}^{i} \right)} = {{S\left( P_{A,A}^{i} \right)} + \frac{c\left( P_{A,A}^{i} \right)}{\varphi_{A}} + {\frac{d\left( {P_{A,A}\left( T_{n + 1} \right)} \right)}{\varphi_{A}}.}}$

For meanings of the virtual start time S(P^(i) _(A,A)), the virtual finish time F(P^(i) _(A,A)), and the virtual time v(P^(i) _(A,A)), refer to a Start-time Fairness Queuing (SFQ) algorithm, and details are not described in this embodiment of the present invention again. c(P^(i) _(A,A)) indicates a resource that is provided by the storage array A and consumed by the P^(i) _(A,A), which is one IOPS in this embodiment of the present invention; and d(P_(A,A)(T_(n+1))) indicates a value of a delay of each user A request that is received by the scheduler A in a T_(n+1) period, and

${d\left( {P_{A,A}\left( T_{n + 1} \right)} \right)} = {\frac{{C_{A}\left( T_{n} \right)} - {C_{A,A}\left( T_{n} \right)}}{N_{A,A}\left( T_{n} \right)} = {\frac{C_{A,B}\left( T_{n} \right)}{N_{A,A}\left( T_{n} \right)}.}}$

Likewise, a virtual start time of a P^(k) _(B,A) that is received by the scheduler A is represented by S(P^(k) _(B,A)), and a virtual finish time of the P^(k) _(B,A) is represented by F(P^(k) _(B,A)). S(P^(k) _(B,A))=max{v(P^(k) _(B,A)), F(P^(k−1) _(B,A))}, where v(P^(k) _(B,A)) indicates a virtual time of the scheduler A when the scheduler A receives the P^(k) _(B,A), F(P^(k−1) _(B,A)) indicates a virtual finish time of a P^(k−1) _(B,A), and max{v(P^(k) _(B,A)), F(P^(k−1) _(B,A))} indicates a maximum value in v(P^(k) _(B,A)) and

${{{F\left( P_{B,A}^{k - 1} \right)}.\mspace{14mu} {F\left( P_{B,A}^{k} \right)}} = {{S\left( P_{B,A}^{k} \right)} + \frac{c\left( P_{B,A}^{k} \right)}{\varphi_{B}} + \frac{d\left( {P_{B,A}\left( T_{n + 1} \right)} \right)}{\varphi_{B}}}},$

where c(P^(k) _(B,A)) indicates a resource that is provided by the storage array B and consumed by the P^(k) _(B,A), which is one IOPS in this embodiment of the present invention; and d(P_(B,A)(T_(n+1))) indicates a value of a delay of each user B request that is received by the scheduler A in a T_(n+1) period, and

$\begin{matrix} {{d\left( {P_{B,A}\left( T_{n + 1} \right)} \right)} = \frac{{C_{B}\left( T_{n} \right)} - {C_{B,A}\left( T_{n} \right)}}{N_{B,A}\left( T_{n} \right)}} \\ {= {\frac{C_{B,B}\left( T_{n} \right)}{N_{B,A}\left( T_{n} \right)}.}} \end{matrix}$

A virtual start time of a P^(i) _(A,B) that is received by the scheduler B is represented by S(P^(i) _(A,B)), and a virtual finish time of the P^(i) _(A,B) is represented by F(P^(i) _(A,B)). S(P^(i) _(A,B))=max{v(P^(i) _(A,B)), F(P^(i−1) _(A,B))}, where v(P^(i) _(A,B)) indicates a virtual time of the scheduler B when the scheduler B receives the P^(i) _(A,B), F(P^(i−1) _(A,B)) indicates a virtual finish time of a P^(i−1) _(A,B), and max{v(P^(i) _(A,B)), F(P^(i−1) _(A,B))} indicates a maximum value in v(P^(i) _(A,B)) and F(P^(i−1) _(A,B)).

${{F\left( P_{A,B}^{i} \right)} = {{S\left( P_{A,B}^{i} \right)} + \frac{c\left( P_{A,B}^{i} \right)}{\varphi_{A}} + \frac{d\left( {P_{A,B}\left( T_{n + 1} \right)} \right)}{\varphi_{A}}}},$

where c(P^(i) _(A,B)) indicates a resource that is provided by the storage array B and consumed by the P^(i) _(A,B), which is one IOPS in this embodiment of the present invention; and d(P_(A,B)(T_(n+1))) indicates a value of a delay of each user A request that is received by the scheduler B in a T_(n+1) period, and

$\begin{matrix} {{d\left( {P_{A,B}\left( T_{n + 1} \right)} \right)} = \frac{{C_{A}\left( T_{n} \right)} - {C_{A,B}\left( T_{n} \right)}}{N_{A,B}\left( T_{n} \right)}} \\ {= {\frac{C_{A,A}\left( T_{n} \right)}{N_{A,B}\left( T_{n} \right)}.}} \end{matrix}$

Likewise, a virtual start time of a P^(k) _(B,B) that is received by the scheduler B is represented by S(P^(k) _(B,B)), and a virtual finish time of the P^(k) _(B,B) is represented by F(P^(k) _(B,B)). S(P^(k) _(B,B))=max{v(P^(k) _(B,B)), F(P^(K−1) _(B,B))}, where v(P^(k) _(B,B)) indicates a virtual time of the scheduler B when the scheduler B receives the P^(i) _(B,B), F(P^(k−1) _(B,B)) indicates a virtual finish time of a P^(k−1) _(B,B), and max{v(P^(k) _(B,B)), F(P^(k−1) _(B,B))} indicates a maximum value in v(P^(k) _(B,B)) and F(P^(k−1) _(B,B)).

${{F\left( P_{B,B}^{k} \right)} = {{S\left( P_{B,B}^{i} \right)} + \frac{c\left( P_{B,B}^{k} \right)}{\varphi_{B}} + \frac{d\left( {P_{B,B}\left( T_{n + 1} \right)} \right)}{\varphi_{B}}}},$

where c(P^(k) _(B,B)) indicates a resource that is provided by the storage array B and consumed by the P^(k) _(B,B), which is one IOPS in this embodiment of the present invention; and d(P_(B,B)(T_(n+1))) indicates a value of a delay of each user B request that is received by the scheduler B in a T_(n+1) period, and

$\begin{matrix} {{d\left( {P_{B,B}\left( T_{n + 1} \right)} \right)} = \frac{{C_{B}\left( T_{n} \right)} - {C_{B,B}\left( T_{n} \right)}}{N_{B,B}\left( T_{n} \right)}} \\ {= {\frac{C_{B,A}\left( T_{n} \right)}{N_{B,B}\left( T_{n} \right)}.}} \end{matrix}$

In a second implementation manner, a virtual start time of a P^(i) _(A,A) that is received by the scheduler A is represented by S(P^(i) _(A,A)), and a virtual finish time of the P^(i) _(A,A) is represented by

${{{F\left( P_{A,A}^{i} \right)} \cdot {S\left( P_{A,A}^{i} \right)}} = {\max \left\{ {{v\left( P_{A,A}^{i} \right)},{{F\left( P_{A,A}^{i - 1} \right)} + \frac{d\left( {P_{A,A}\left( T_{n + 1} \right)} \right)}{\varphi_{A}}}} \right\}}},$

where v(P^(i) _(A,A)) indicates a virtual time of the scheduler A when the scheduler A receives the P^(i) _(A,A); d(P_(A,A)(T_(n+1))) indicates a value of a delay of each user A request that is received by the scheduler A in a T_(n+1) period, and

$\begin{matrix} {{d\left( {P_{A,A}\left( T_{n + 1} \right)} \right)} = \frac{{C_{A}\left( T_{n} \right)} - {C_{A,A}\left( T_{n} \right)}}{N_{A,A}\left( T_{n} \right)}} \\ {{= \frac{C_{A,B}\left( T_{n} \right)}{N_{A,A}\left( T_{n} \right)}};} \end{matrix}$

F(P^(i−1) _(A,A)) indicates a virtual finish time of a P^(i−1) _(A,A); and

$\max \left\{ {{v\left( P_{A,A}^{i} \right)},{{F\left( P_{A,A}^{i - 1} \right)} + \frac{d\left( {P_{A,A}\left( T_{n + 1} \right)} \right)}{\varphi_{A}}}} \right\}$

indicates a maximum value in v(P^(i) _(A,A)) and

${{F\left( P_{A,A}^{i - 1} \right)} + {\frac{d\left( {P_{A,A}\left( T_{n + 1} \right)} \right)}{\varphi_{A}} \cdot {F\left( P_{A,A}^{i} \right)}}} = {{S\left( P_{A,A}^{i} \right)} + {\frac{c\left( P_{A,A}^{i} \right)}{\varphi_{A}}.}}$

For meanings of the virtual start time S(P^(i) _(A,A)), the virtual finish time F(P^(i) _(A,A)) and the virtual time v(P^(i) _(A,A)), refer to a Start-time Fairness Queuing (SFQ) algorithm, and details are not described in this embodiment of the present invention again. c(P^(i) _(A,A)) indicates a resource that is provided by the storage array A and consumed by the P^(i) _(A,A), which is one IOPS in this embodiment of the present invention. Likewise, a virtual start time of a P^(k) _(B,A) that is received by the scheduler A is represented by S(P^(k) _(B,A)), and a virtual finish time of the P^(k) _(B,A) is represented by F(P^(k) _(B,A)).

${{S\left( P_{B,A}^{k} \right)} = {\max \left\{ {{v\left( P_{B,A}^{k} \right)},{{F\left( P_{B,A}^{k - 1} \right)} + \frac{d\left( {P_{B,A}\left( T_{n + 1} \right)} \right)}{\varphi_{B}}}} \right\}}},$

where v(P^(k) _(B,A)) indicates a virtual time of the scheduler A when the scheduler A receives the P^(k) _(B,A); d(P_(B,A)(T)) indicates a value of a delay of each user B request that is received by the scheduler A in a T_(n+1) period, and

$\begin{matrix} {{d\left( {P_{B,A}\left( T_{n + 1} \right)} \right)} = \frac{{C_{B}\left( T_{n} \right)} - {C_{B,A}\left( T_{n} \right)}}{N_{B,A}\left( T_{n} \right)}} \\ {{= \frac{C_{B,B}\left( T_{n} \right)}{N_{B,A}\left( T_{n} \right)}};} \end{matrix}$

F(P^(k−1) _(B,A)) indicates a virtual finish time of a P^(k−1) _(B,A); and

$\max \left\{ {{v\left( P_{B,A}^{k} \right)},{{F\left( P_{B,A}^{k - 1} \right)} + \frac{d\left( {P_{B,A}\left( T_{n + 1} \right)} \right)}{\varphi_{B}}}} \right\}$

indicates a maximum value in v(P^(k) _(B,A)) and

${{{F\left( P_{B,A}^{k - 1} \right)} + {\frac{d\left( {P_{B,A}\left( T_{n + 1} \right)} \right)}{\varphi_{B}} \cdot {F\left( P_{B,A}^{k} \right)}}} = {{S\left( P_{B,A}^{k} \right)} + \frac{c\left( P_{B,A}^{k} \right)}{\varphi_{B}}}},$

where c(P^(k) _(B,A)) indicates a resource that is provided by the storage array B and consumed by the P^(k) _(B,A), which is one IOPS in this embodiment of the present invention.

A virtual start time of a P^(i) _(A,B) that is received by the scheduler B is represented by S(P^(i) _(A,B)), and a virtual finish time of the P^(i) _(A,B) is represented by F(P^(i) _(A,B)).

${{S\left( P_{A,B}^{i} \right)} = {\max \left\{ {{v\left( P_{A,B}^{i} \right)},{{F\left( P_{A,B}^{i - 1} \right)} + \frac{d\left( {P_{A,B}\left( T_{n + 1} \right)} \right)}{\varphi_{A}}}} \right\}}},$

where v(P^(i) _(A,B)) indicates a virtual time of the scheduler B when the scheduler B receives the P^(i) _(A,B); d(P_(A,B)(T_(n+1))) indicates a value of a delay of each user A request that is received by the scheduler B in a T_(n+1) period, and

$\begin{matrix} {{d\left( {P_{A,B}\left( T_{n + 1} \right)} \right)} = \frac{{C_{A}\left( T_{n} \right)} - {C_{A,B}\left( T_{n} \right)}}{N_{A,B}\left( T_{n} \right)}} \\ {{= \frac{C_{A,A}\left( T_{n} \right)}{N_{A,B}\left( T_{n} \right)}};} \end{matrix}$

F(P^(i−1) _(A,B)) indicates a virtual finish time of a P^(i−1) _(A,B); and

$\max \left\{ {{v\left( P_{A,B}^{i} \right)},{{F\left( P_{A,B}^{i - 1} \right)} + \frac{d\left( {P_{A,B}\left( T_{n + 1} \right)} \right)}{\varphi_{A}}}} \right\}$

indicates a maximum value in v(P^(i) _(A,B)) and

${{{F\left( P_{A,B}^{i - 1} \right)} + {\frac{d\left( {P_{A,B}\left( T_{n + 1} \right)} \right)}{\varphi_{A}} \cdot {F\left( P_{A,B}^{i} \right)}}} = {{S\left( P_{A,B}^{i} \right)} + \frac{c\left( P_{A,B}^{i} \right)}{\varphi_{A}}}},$

where c(P^(i) _(A,B)) indicates a resource that is provided by the storage array B and consumed by the P^(i) _(A,B) which is one IOPS in this embodiment of the present invention. Likewise, a virtual start time of a P^(k) _(B,B) that is received by the scheduler B is represented by S(P^(k) _(B,B)), and a virtual finish time of the P^(k) _(B,B) is represented by

${{{F\left( P_{B,B}^{k} \right)} \cdot {S\left( P_{B,B}^{k} \right)}} = {\max \left\{ {{v\left( P_{B,B}^{k} \right)},{{F\left( P_{B,B}^{k - 1} \right)} + \frac{d\left( {P_{B,B}\left( T_{n + 1} \right)} \right)}{\varphi_{B}}}} \right\}}},$

where v(P^(k) _(B,B)) indicates a virtual time of the scheduler B when the scheduler B receives the P^(k) _(B,B); d(P_(B,B)(T_(n+1))) indicates a value of a delay of each user B request received by the scheduler A in a T_(n+1) period, and

$\begin{matrix} {{d\left( {P_{B,B}\left( T_{n + 1} \right)} \right)} = \frac{{C_{B}\left( T_{n} \right)} - {C_{B,B}\left( T_{n} \right)}}{N_{B,B}\left( T_{n} \right)}} \\ {{= \frac{C_{B,A}\left( T_{n} \right)}{N_{B,B}\left( T_{n} \right)}};} \end{matrix}$ F(P_(B, B)^(k − 1))

indicates a virtual finish time of a P^(k−1) _(B,B); and

$\max \left\{ {{v\left( P_{B,B}^{k} \right)},{{F\left( P_{B,B}^{k - 1} \right)} + \frac{d\left( {P_{B,B}\left( T_{n + 1} \right)} \right)}{\varphi_{B}}}} \right\}$

indicates a maximum value in v(P^(k) _(B,B)) and

${{{F\left( P_{B,B}^{k - 1} \right)} + {{\frac{d\left( {P_{B,B}\left( T_{n + 1} \right)} \right)}{\varphi_{B}} \cdot F}\left( P_{B,B}^{k} \right)}} = {{S\left( P_{B,B}^{i} \right)} + \frac{c\left( P_{B,B}^{k} \right)}{\varphi_{B}}}},$

where c(P^(k) _(B,B)) indicates a resource that is provided by the storage array B and consumed by the P^(k) _(B,B) which is one IOPS in this embodiment of the present invention.

The first manner of computing a virtual start time and a virtual finish time of a user request is used as an example. As shown in FIG. 5, in this embodiment, after a distributed resource system is initialized, user A requests P^(i) _(A,A) that are received by a scheduler A in a T_(n+1) period are P¹ _(A,A), P² _(A,A), P³ _(A,A), P⁴ _(A,A), and P⁵ _(A,A) respectively, 5 in total. A quantity of user A requests P^(i) _(A,B) that are received by a scheduler B is 0. User B requests P^(k) _(B,A) that are received by the scheduler A are P¹ _(A,B), P² _(B,A), P³ _(B,A), and P⁴ _(B,A) respectively, 4 in total. User B requests P^(k) _(B,B) that are received by the scheduler B are P¹ _(B,B), P² _(B,B), P³ _(B,B), P⁴ _(B,B), and P⁵ _(B,B) respectively, 5 in total. In this embodiment of the present invention, the T_(n+1) period is actually the first period. The P¹ _(A,A) is the first user A request received by the scheduler A. In this case, the received P¹ _(A,A) needs to be added to a scheduling queue of the scheduler A for scheduling according to a scheduling algorithm. First, a virtual start time S(P¹ _(A,A))=max{v(P¹ _(A,A)), F(P^(i−1) _(A,A))} of the P¹ _(A,A) is computed. Because the T_(n+1) period is actually the first period, that is, the scheduler A does not receive any P^(i) _(A,A) previously, F(P⁰ _(A,A)) is Null, the P¹ _(A,A) is received after initialization of the scheduler A is completed, and S(P¹ _(A,A))=v(P¹ _(A,A))=0. A virtual finish time of the P¹ _(A,A) is computed:

${{F\left( P_{A,A}^{1} \right)} = {{S\left( P_{A,A}^{1} \right)} + \frac{c\left( P_{A,A}^{1} \right)}{\varphi_{A}} + \frac{d\left( {P_{A,A}\left( T_{n + 1} \right)} \right)}{\varphi_{A}}}},{where}$ ${{S\left( P_{A,A}^{1} \right)} = 0},{{c\left( P_{A,A}^{i} \right)} = 1},{\varphi_{A} = 2},\begin{matrix} {{d\left( {P_{A,A}\left( T_{n + 1} \right)} \right)} = \frac{{C_{A}\left( T_{n} \right)} - {C_{A,A}\left( T_{n} \right)}}{N_{A,A}\left( T_{n} \right)}} \\ {{= \frac{C_{A,B}\left( T_{n} \right)}{N_{A,A}\left( T_{n} \right)}},} \end{matrix}$ C_(A, B)(T_(n)) = 0, and N_(A, A)(T_(n)) = 0;

then d(P_(A,A)(T_(n+1)))=0, and F(P¹ _(A,A))=S(P¹ _(A,A))+½=0.5. Likewise, a virtual start time of the P² _(A,A) is 0.5, and a virtual finish time of the P² _(A,A) is 1; a virtual start time of the P³ _(A,A) is 1, and a virtual finish time of the P³ _(A,A) is 1.5; a virtual start time of the P⁴ _(A,A) is 1.5, and a virtual finish time of the P⁴ _(A,A) is 2; and a virtual start time of the P⁵ _(A,A) is 2, and a virtual finish time of the P⁵ _(A,A) is 2.5.

$P_{A,A}^{1}\begin{matrix} (0.5) \\ (0) \end{matrix}$

in FIG. 5 indicates that the virtual start time of the P¹ _(A,A) is 0 and the virtual finish time of the P¹ _(A,A) is 0.5. For other similar identifiers in FIG. 5 and subsequent figures, refer to the meaning of

${P_{A,A}^{1}\begin{matrix} (0.5) \\ (0) \end{matrix}},$

and details are not described herein again.

For the user B request P¹ _(B,A) that is received by the scheduler A, a virtual start time of the P¹ _(B,A) is computed: S(P¹ _(B,A))=max{v(P¹ _(B,A)), F(P⁰ _(B,A))}. Because the T_(n+1) period is actually the first period, that is, the scheduler A does not receive any P^(k) _(B,A) previously, F(P⁰ _(B,A)) is empty, the P¹ _(B,A) is received after initialization of the scheduler A is completed, and v(P¹ _(B,A))=0. A virtual finish time of the P¹ _(B,A) is

${{F\left( P_{B,A}^{1} \right)} = {{S\left( P_{B,A}^{1} \right)} + \frac{c\left( P_{B,A}^{1} \right)}{\varphi_{B}} + \frac{d\left( {P_{B,A}\left( T_{n + 1} \right)} \right)}{\varphi_{B}}}},$

where

${{c\left( P_{B,A}^{1} \right)} = 1},\begin{matrix} {{d\left( {P_{B,A}\left( T_{n + 1} \right)} \right)} = \frac{{C_{B}\left( T_{n} \right)} - {C_{B,A}\left( T_{n} \right)}}{N_{B,A}\left( T_{n} \right)}} \\ {{= \frac{C_{B,B}\left( T_{n} \right)}{N_{B,A}\left( T_{n} \right)}},} \end{matrix}$

C_(B,B)(T_(n))=0, and N_(B,A)(T_(n))=0; then d(P_(B,A)(T_(n+1)))=0, φ_(B)=1, and F(P¹ _(B,A))=1. Likewise, a virtual start time of the P² _(B,A) is 1, and a virtual finish time of the P² _(B,A) is 2; a virtual start time of the P³ _(B,A) is 2, and a virtual finish time of the P³ _(B,A) is 3; and a virtual start time of the P⁴ _(B,A) is 3, and a virtual finish time of the P⁴ _(B,A) is 4.

In this case, for the user B request P¹ _(B,B) that is received by the scheduler B, the received P¹ _(B,B) needs to be scheduled, and a virtual start time of the P¹ _(B,B) is computed: S(P¹ _(B,B))=max{v(P¹ _(B,B)), F(P⁰ _(B,B))}. Because the T_(n+1) period is actually the first period, that is, the scheduler B does not receive any P^(k) _(B,B) previously, F(P⁰ _(B,B)) is empty, the P¹ _(B,B) is received after initialization of the scheduler B is completed, and v(P¹ _(B,B))=0. A virtual finish time of the P¹ _(B,B) is

${{F\left( P_{B,B}^{1} \right)} = {{S\left( P_{B,B}^{1} \right)} + \frac{c\left( P_{B,B}^{1} \right)}{\varphi_{B}} + \frac{d\left( {P_{B,B}\left( T_{n + 1} \right)} \right)}{\varphi_{B}}}},$

where

${{c\left( P_{B,B}^{1} \right)} = 1},\begin{matrix} {{d\left( {P_{B,B}\left( T_{n + 1} \right)} \right)} = \frac{{C_{B}\left( T_{n} \right)} - {C_{B,B}\left( T_{n} \right)}}{N_{B,B}\left( T_{n} \right)}} \\ {{= \frac{C_{B,A}\left( T_{n} \right)}{N_{B,B}\left( T_{n} \right)}},} \end{matrix}$

C_(B,A)(T_(n))=0, and N_(B,B)(T_(n))=0; then d(P_(B,B)(T_(n+1)))=0, φ_(B)=1, and F(P¹ _(B,B))=1. Likewise, a virtual start time of the P² _(B,B) is 1, and a virtual finish time of the P² _(B,B) is 2; a virtual start time of the P³ _(B,B) is 2, and a virtual finish time of the P³ _(B,B) is 3; a virtual start time of the P⁴ _(B,B) is 3, and a virtual finish time of the P⁴ _(B,B) is 4; and a virtual start time of the P⁵ _(B,B) is 4, and a virtual finish time of the P⁵ _(B,B) is 5.

The scheduler A computes virtual start times and virtual finish times of the P¹ _(A,A), P² _(A,A), P³ _(A,A), P⁴ _(A,A), and P⁵ _(A,A), and computes virtual start times and virtual finish times of the P¹ _(B,A), P² _(B,A), P³ _(B,A), and P⁴ _(B,A). According to the virtual start times of the user requests that are received by the scheduler A, the user requests are ranked in a scheduling queue in ascending order of the virtual start times for processing, that is, an order from the queue head to the queue tail in the scheduling queue is P¹ _(A,A), P² _(A,A), P¹ _(B,A), P³ _(A,A), P⁴ _(A,A), P² _(B,A), P⁵ _(A,A), P³ _(B,A), and P⁴ _(B,A), as shown in FIG. 6. The storage array A processes the user requests in the order of P¹ _(A,A), P² _(A,A), P¹ _(B,A), P³ _(A,A), P⁴ _(A,A), P² _(B,A), P⁵ _(A,A), P³ _(B,A), and P⁴ _(B,A). As shown in FIG. 6, when there is resource competition between the user A requests and the user B requests, a group of user requests P¹ _(A,A), P² _(A,A), and P¹ _(B,A) in the scheduling queue consume, according to a ratio of 2:1, the IOPS that is provided by the storage array A. Another group of user requests P³ _(A,A), P⁴ _(A,A), and P² _(B,A) in the scheduling queue also meets the requirement.

The scheduler B computes virtual start times and virtual finish times of the P¹ _(B,B), P² _(B,B), P³ _(B,B), P⁴ _(B,B), and P⁵ _(B,B) respectively. According to the virtual start times of all the user requests that are received by the scheduler B, the user requests are ranked in a scheduling queue in ascending order of the virtual start times for processing, that is, an order from the queue head to the queue tail in the scheduling queue is P¹ _(B,B), P² _(B,B), P³ _(B,B), P⁴ _(B,B), and P⁵ _(B,B), as shown in FIG. 7. The storage array B processes the user requests in the order of P¹ _(B,B), P² _(B,B), P³ _(B,B), P⁴ _(B,B), and P⁵ _(B,B). Because there are only the user B requests in the scheduler B, there is no resource competition, and the user B requests are processed in ascending order of the virtual start times of the user B requests.

A person skilled in the art should understand that, in specific implementation, a quantity of user requests that are received by the scheduler A and the scheduler B in a T_(n+1) period may be much greater than the quantity of the user requests provided in the embodiment. A quantity of users is not limited to two. For ease of description in this embodiment of the present invention, in the T_(n+1) period, the user A requests that are received by the scheduler A are P¹ _(A,A), P² _(A,A), P³ _(A,A), P⁴ _(A,A), and P⁵ _(A,A), and the user B requests that are received by the scheduler A are P¹ _(B,A), P² _(B,A), P³ _(B,A), and P⁴ _(B,A); and the user B requests that are received by the scheduler B are P¹ _(B,B), P² _(B,B), P³ _(B,B), P⁴ _(B,B), and P⁵ _(B,B).

According to the manner described above, the scheduler A computes a resource C_(A,A)(T_(n+1)) that is provided by the storage array A and consumed by N_(A,A)(T_(n+1)) user A requests received in the T_(n+1) period, and a resource C_(B,A)(T_(n)) that is provided by the storage array A and consumed by N_(B,A)(T_(n)) received user B requests. The scheduler A computes a resource C_(A,A)(T_(n+1)) that is provided by the storage array A and consumed by N_(A,A)(T_(n+1)) user A requests received in the T_(n+1) period, which is specifically that: in this embodiment of the present invention, one user A request consumes one IOPS, and a quantity of the user A requests received by the scheduler A in the T_(n+1) period is N_(A,A)(T_(n+1))=5, and the consumed resource C_(A,A)(T_(n+1)) that is provided by the storage array A is five IOPSs. Likewise, a quantity of the user B requests that are received by the scheduler A in the T_(n+1) period is N_(B,A)(T_(n+1))=4, and the consumed resource C_(B,A)(T_(n+1)) that is provided by the storage array A is four IOPSs. A quantity of the user A requests that are received by the scheduler B in the T_(n+1) period is N_(A,B)(T_(n+1))=0, and the consumed resource C_(A,B)(T_(n+1)) that is provided by the storage array B is 0 IOPS. Likewise, a quantity of the user B requests that are received by the scheduler B in the T_(n+1) period is N_(B,B)(T_(n+1))=5, and the consumed resource C_(B,B)(T_(n+1)) that is provided by the storage array B is five IOPSs. The coordinator A acquires that the resource C_(A,A)(T_(n+1)) that is provided by the storage array A and consumed by the user A requests processed by the scheduler A in the T_(n+1) period is five IOPSs, and the resource that is provided by the storage array B and consumed by the user A requests processed by the scheduler B in the T_(n+1) period is zero IOPS. Therefore, a total quantity of resources that are consumed by the user A requests in the T_(n+1) period is C_(A)(T_(n+1))=C_(A,A)(T_(n+1))+C_(A,B)(T_(n+1)), where C_(A)(T_(n+1))=5 IOPSs. Likewise, the coordinator B acquires that a total quantity of the resources that are consumed by the user B requests in the T_(n+1) period is C_(B)(T_(n+1))=C_(B,A)(T_(n+1))+C_(B,B)(T_(n+1)), where C_(B)(T_(n+1))=9 IOPSs. The coordinator A sends a message to each of the scheduler A and the scheduler B, where the message carries C_(A)(T_(n+1)), so that the scheduler A and the scheduler B acquire, from the coordinator A, that the quantity of IOPSs that are consumed by the user A requests in the T_(n+1) period is 5. Likewise, the coordinator B sends a message to each of the scheduler A and the scheduler B, where the message carries C_(B)(T_(n+1)), so that the scheduler A and the scheduler B acquire, from the coordinator B, that the quantity of IOPSs that are consumed by the user B requests in the T_(n+1) period is 9.

As shown in FIG. 8, in a T_(n+2) period, the scheduler A receives the first user A request P⁶ _(A,A), where a virtual start time of the P⁶ _(A,A) is S(P⁶ _(A,A))=max{v(P⁶ _(A,A)), F(P⁵ _(A,A))}, where v(P⁶ _(A,A))=F(P⁵ _(A,A))=2.5, and therefore,

${{S\left( P_{A,A}^{6} \right)} = {{{2.5 \cdot F}\left( P_{A,A}^{6} \right)} = {{S\left( P_{A,A}^{6} \right)} + \frac{c\left( P_{A,A}^{6} \right)}{\varphi_{A}} + \frac{d\left( {P_{A,A}\left( T_{n + 2} \right)} \right)}{\varphi_{A}}}}},$

where

$\begin{matrix} {{d\left( {P_{A,A}\left( T_{n + 2} \right)} \right)} = \frac{{C_{A}\left( T_{n + 1} \right)} - {C_{A,A}\left( T_{n + 1} \right)}}{N_{A,A}\left( T_{n + 1} \right)}} \\ {{= \frac{C_{A,B}\left( T_{n + 1} \right)}{N_{A,A}\left( T_{n + 1} \right)}},} \end{matrix}$

and C_(A,B)(T_(n+1))=0 and N_(A,A)(T_(n+1))=5 in the T_(n+1) period; then d(P_(A,A)(T_(n+2)))=0; c(P⁶ _(A,A))=1, φ_(A)=2, and therefore, F(P⁶ _(A,A))=3. Likewise, a virtual start time of a P⁷ _(A,A) is 3, and a virtual finish time of the P⁷ _(A,A) is 3.5; a virtual start time of a P⁸ _(A,A) is 3.5, and a virtual finish time of the P⁸ _(A,A) is 4; and a virtual start time of a P⁹ _(A,A) is 4, and a virtual finish time of the P⁹ _(A,A) is 4.5.

In the T_(n+2) period, the scheduler A receives the first user B request P⁵ _(B,A), where S(P⁵ _(B,A))=max{v(P⁵ _(B,A)), F(P⁴ _(B,A))}, v(P⁵ _(B,A))=F(P⁴ _(B,A))=4, and therefore, S(P⁵ _(B,A))=4. A virtual finish time of the P⁵ _(B,A) is

${{F\left( P_{B,A}^{5} \right)} = {{S\left( P_{B,A}^{5} \right)} + \frac{c\left( P_{B,A}^{5} \right)}{\varphi_{B}} + \frac{d\left( {P_{B,A}\left( T_{n + 2} \right)} \right)}{\varphi_{B}}}},$

where c(P⁵ _(B,A))=1,

$\begin{matrix} {{d\left( {P_{B,A}\left( T_{n + 2} \right)} \right)} = \frac{{C_{B}\left( T_{n + 1} \right)} - {C_{B,A}\left( T_{n + 1} \right)}}{N_{B,A}\left( T_{n + 1} \right)}} \\ {{= \frac{C_{B,B}\left( T_{n + 1} \right)}{N_{B,A}\left( T_{n + 1} \right)}},} \end{matrix}$

C_(B,B)(T_(n+1))=5, N_(B,A)(T_(n+1))=4, d(P_(B,A)(T_(n+2)))=1.25, φ_(B)=1, and therefore, F(P⁵ _(B,A))=6.25. Likewise, a virtual start time of a P⁶ _(B,A) is 6.25, and a virtual finish time of the P⁶ _(B,A) is 8.5; a virtual start time of a P⁷ _(B,A) is 8.5, and a virtual finish time of the P⁷ _(B,A) is 10.75; and a virtual start time of a P⁸ _(B,A) is 10.75, and a virtual finish time of the P⁸ _(B,A) is 13.

The scheduler A computes virtual start times and virtual finish times of the P⁶ _(A,A), P⁷ _(A,A), P⁸ _(A,A), and P⁹ _(A,A), and the scheduler A computes virtual start times and virtual finish times of the P⁵ _(B,A), P⁶ _(B,A), P⁷ _(B,A), and P⁸ _(B,A). According to the virtual start times of the user requests that are received by the scheduler A, the user requests are ranked in a scheduling queue in ascending order of the virtual start times for processing, that is, an order from the queue head to the queue tail in the scheduling queue is P⁶ _(A,A), P⁷ _(A,A), P⁸ _(A,A), P⁹ _(A,A), P⁵ _(B,A), P⁶ _(B,A), P⁷ _(B,A), and P⁸ _(B,A), as shown in FIG. 9. The storage array A processes the user requests in the order of P⁶ _(A,A), P⁷ _(A,A), P⁸ _(A,A), P⁹ _(A,A), P⁵ _(B,A), P⁶ _(B,A), P⁷ _(B,A), and P⁸ _(B,A).

In the T_(n+2) period, the scheduler B receives the first user A request P¹ _(A,B) and the first user B request P⁶ _(B,B). It is assumed that the scheduler B receives the P¹ _(A,B) and the P⁶ _(B,B) at a same moment. A virtual start time of the P⁶ _(B,B) is S(P⁶ _(B,B))=max{v(P⁶ _(B,B)), F(P⁵ _(B,B))}, where v(P⁶ _(B,B))=F(P⁵ _(B,B))=5, and therefore, S(P⁶ _(B,B))=5. A virtual finish time of the P⁶ _(B,B) is

${{F\left( P_{B,B}^{6} \right)} = {{S\left( P_{B,B}^{6} \right)} + \frac{c\left( P_{B,B}^{6} \right)}{\varphi_{B}} + \frac{d\left( {P_{B,B}\left( T_{n + 2} \right)} \right)}{\varphi_{B}}}},$

where

${{c\left( P_{B,B}^{6} \right)} = 1},{{d\left( {P_{B,B}\left( T_{n + 2} \right)} \right)} = {\frac{{C_{B}\left( T_{n + 1} \right)} - {C_{B,B}\left( T_{n + 1} \right)}}{N_{B,B}\left( T_{n + 1} \right)} = {\frac{C_{B,A}\left( T_{n + 1} \right)}{N_{B,B}\left( T_{n + 1} \right)} = {\frac{4}{5} = 0.8}}}},$

and φ_(B)=1, and therefore, F(P⁶ _(B,B))=6.8. Likewise, a virtual start time of a P⁷ _(B,B) is 6.8, and a virtual finish time of the P⁷ _(B,B) is 8.6; a virtual start time of a P⁸ _(B,B) is 8.6, and a virtual finish time of the P⁸ _(B,B) is 10.4. A virtual start time of the P¹ _(A,B) is S(P¹ _(A,B))=max{v(P¹ _(A,B)), F(P⁰ _(A,B))}, where F(P⁰ _(A,B))=0 and v(P¹ _(A,B))=v(P⁶ _(B,B))=5, and therefore, S(P¹ _(A,B))=5. A virtual finish time of the P¹ _(A,B) is

${{F\left( P_{A,B}^{1} \right)} = {{S\left( P_{A,B}^{1} \right)} + \frac{c\left( P_{A,B}^{1} \right)}{\varphi_{B}} + \frac{d\left( {P_{A,B}\left( T_{n + 2} \right)} \right)}{\varphi_{B}}}},$

where c(P¹ _(A,B))=1,

$\begin{matrix} {{d\left( {P_{A,B}\left( T_{n + 2} \right)} \right)} = \frac{{C_{A}\left( T_{n + 1} \right)} - {C_{A,B}\left( T_{n + 1} \right)}}{N_{A,B}\left( T_{n + 1} \right)}} \\ {{= \frac{C_{A,A}\left( T_{n + 1} \right)}{N_{A,B}\left( T_{n + 1} \right)}},} \end{matrix}$

C_(A,A)(T_(n+1))=5, and N_(A,A)(T_(n+1))=0; then d(P_(A,B)(T_(n+2)))=0, and φ_(A)=2, and therefore, F(P¹ _(A,B))=5.5. Likewise, a virtual start time of a P² _(A,B) is 5.5, and a virtual finish time of the P² _(A,B) is 6; and a virtual start time of a P³ _(A,B) is 6, and a virtual finish time of the P³ _(A,B) is 6.5.

The scheduler B adds the user requests to a scheduling queue, and the user requests are ranked in the scheduling queue in ascending order of the virtual start times, that is, an order from the queue head to the queue tail in the scheduling queue is P¹ _(A,B), P⁶ _(B,B), P² _(A,B), P³ _(A,B), P⁷ _(B,B), and P⁸ _(B,B), as shown in FIG. 10. The storage array B processes the user requests in the order of P¹ _(A,B), P⁶ _(B,B), P² _(A,B), P³ _(A,B), P⁷ _(B,B), and P⁸ _(B,B).

The method for scheduling a user request in a distributed resource system provided by this embodiment of the present invention may be further applied to a scenario shown in FIG. 11, where the distributed resource system includes a user 1 to a user H, a scheduler S₁ to a scheduler S_(M), a resource-providing entity R₁ to a resource-providing entity R_(M), and a coordinator G₁ to a coordinator G_(Y), where H is a natural number greater than or equal to 2, M is a natural number greater than or equal to 2, and Y is a natural number. A scheduler S_(x) communicates with a resource-providing entity R_(x), where x is any natural number from 1 to M. Any user z can send a user z request to any scheduler S_(x). The R_(x) provides a resource for a user request received by the S_(x). The coordinator G_(y) communicates with any R_(x), where y is any natural number from 1 to Y, and a value of Y may be less than or equal to M, or may be greater than M. Y may also be 1, that is, there is only one coordinator in the distributed resource system. A quantity of coordinators may be determined according to a quantity of schedulers, or may be determined according to a quantity of users, which is not limited in this embodiment of the present invention. Any user z sends the i^(th) user z request to a scheduler S_(d) in a T_(n+1) period, where i is a natural number, d is a natural number, 1≦d≦M, and S_(d) indicates one of S₁ to S_(M). When the S_(d) needs to schedule the received user request, the following steps are performed:

Step 1201. The S_(d) acquires, in the T_(n+1) period and from a coordinator G_(k) of a user z, a resource C_(z)(T_(n)) that is consumed by the user z request in a T_(n) period, where

${{C_{z}\left( T_{n} \right)} = {\sum\limits_{x = 1}^{M}{C_{z,x}\left( T_{n} \right)}}};$

d and k are natural numbers, 1≦d≦M, and 1≦k≦Y; a resource weight of the user z is φ_(z); C_(z,x)(T_(n)) is a quantity of resources that are provided by the R_(x) and consumed by N_(z,x)(T_(n)) user z requests received by the S_(x) in the T_(n) period; z indicates an identifier of the user; and

${C_{z}\left( T_{n} \right)} = {\sum\limits_{x = 1}^{M}{C_{z,x}\left( T_{n} \right)}}$

indicates the sum of resources that are provided by the R₁ to the R_(M) and consumed by user z requests received by the S₁ to the S_(M) in the T_(n) period.

In step 1201, C_(z,x)(T_(n)) specifically includes the sum of resources that are provided by the R_(x) and consumed by N_(z,x)(T_(n)) user z requests received by the S_(x) in the T_(n) period. N_(z,x)(T_(n)) indicates a quantity of user z requests that are received by the S_(x) in the T_(n) period. The S_(x) sends C_(z,x)(T_(n)) to a G_(k), and the G_(k) acquires C_(z)(T_(n)) according to

$\sum\limits_{x = 1}^{M}{{C_{z,x}\left( T_{n} \right)}.}$

A manner of acquiring C_(z)(T_(n)) is that the S_(x) proactively sends C_(z,x)(T_(n)) to the G_(k). Another manner is that the S_(x) receives a G_(k) request, and sends C_(z,x)(T_(n)) to the G_(k) according to the G_(k) request. The G_(k) acquires C_(z,x)(T_(n)) according to

$\sum\limits_{x = 1}^{M}{{C_{z,x}\left( T_{n} \right)}.}$

The S_(x) stores N_(z,x)(T_(n)) and C_(z,x)(T_(n)).

Step 1202. The S_(d) schedules, according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)), a P^(i) _(z,d) by using a scheduling algorithm, where the P^(i) _(z,d) is the i^(th) user z request received by the S_(d), and C_(z,d)(T_(n))is a quantity of resources that are provided by an R_(d) and consumed by N_(z,d)(T_(n)) user z requests received by the S_(d) in the T_(n) period.

The S_(d) schedules, according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)), the P^(i) _(z,d) by using a scheduling algorithm, thereby implementing global scheduling on the user z request and ensuring a performance requirement of the user z. Step 1202 specifically includes:

That the S_(d) schedules, according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)), the P^(i) _(z,d) by using a scheduling algorithm specifically includes that: the S_(d) computes a virtual start time S(P^(i) _(z,d)) and a virtual finish time F(P^(i) _(z,d)) of the P^(i) _(z,d) according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)) and adds the P^(i) _(z,d) to a scheduling queue, where the scheduling queue ranks the user request according to a value of the virtual start time of the user request.

${{S\left( P_{z,d}^{i} \right)} = {\max \left\{ {{v\left( P_{z,d}^{i} \right)},{F\left( P_{z,d}^{i - 1} \right)}} \right\}}},{{F\left( P_{z,d}^{i} \right)} = {{S\left( P_{z,d}^{i} \right)} + \frac{c\left( P_{z,d}^{i} \right)}{\varphi_{z}} + \frac{d\left( {P_{z,d}\left( T_{n + 1} \right)} \right.}{\varphi_{z}}}},{{{and}\text{}{d\left( {P_{z,d}\left( T_{n + 1} \right)} \right)}} = \frac{{C_{z}\left( T_{n} \right)} - {C_{z,d}\left( T_{n} \right)}}{N_{z,d}\left( T_{n} \right)}},$

where v(P^(i) _(z,d)) indicates a virtual time of the S_(d) when the S_(d) receives the P^(i) _(z,d), and c(P^(i) _(z,d)) indicates a resource that is provided by the R_(d) and consumed by the P^(i) _(z,d); or,

${{S\left( P_{z,d}^{i} \right)} = {\max \left\{ {{v\left( P_{z,d}^{i} \right)},{{F\left( P_{z,d}^{i - 1} \right)} + \frac{d\left( {P_{z,d}\left( T_{n + 1} \right)} \right)}{\varphi_{z}}}} \right\}}},{{F\left( P_{z,d}^{i} \right)} = {{S\left( P_{z,d}^{i} \right)} + \frac{c\left( P_{z,d}^{i} \right)}{\varphi_{z}}}},{and}$ ${{d\left( {P_{z,d}\left( T_{n + 1} \right)} \right)} = \frac{{C_{z}\left( T_{n} \right)} - {C_{z,d}\left( T_{n} \right)}}{N_{z,d}\left( T_{n} \right)}},$

where v(P^(i) _(z,d)) indicates a virtual time of the S_(d) when the S_(d) receives the P^(i) _(z,d), and c(P^(i) _(z,d)) indicates a resource that is provided by the R_(d) and consumed by the P^(i) _(z,d).

For meanings of the virtual start time S(P^(i) _(A,A)) the virtual finish time F(P^(i) _(A,A)), and the virtual time v(P^(i) _(A,A)), refer to a Start-time Fairness Queuing (SFQ) algorithm, and details are not described in this embodiment of the present invention again.

d(P_(z,d)(T_(n+1))) indicates a value of a delay of each P^(i) _(z,d) received by the S_(d) in the T_(n+1) period.

When either of C_(z)(T_(n))-C_(z,d)(T_(n)) and N_(z,d)(T_(n)) is 0, d(P_(z,d)(T_(n+1)))=0.

c(P^(i) _(z,d)) may be specifically an IOPS or bandwidth that is consumed by the P^(i) _(z,d).

Step 1201 specifically includes:

the G_(k) is determined according to z % Y=k, where z % Y indicates that a modulo operation is performed on z and Y; or the G_(k) is determined according to Hash(z) % Y=k, where Hash(z) indicates that z is computed by using a hash function, and Hash(z) % Y indicates that a modulo operation is performed on a value and Y, where the value is obtained by computing z by using the hash function.

In the application scenario shown in FIG. 11, the G_(k) is configured to send C_(z)(T_(n)) to each scheduler S_(x).

In the application scenario shown in FIG. 11, the method that the S_(d) uses a scheduling algorithm to schedule the user z request is applicable to scheduling of any user request in the distributed resource system in this embodiment of the present invention. Requests of two or more users may be scheduled in the S_(d).

In the application scenario shown in FIG. 11, the scheduler S_(d) schedules, by using a scheduling algorithm, the user z request P^(i) _(z,d) according to C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)) of the user z in the distributed resource system in the T_(n) period, and with reference to the weight φ_(z) of the user, thereby implementing global scheduling on the user z request and ensuring a performance requirement of the user z.

In a system architecture in this embodiment of the present invention, the user z request can be scheduled without depending on a user agent.

In this embodiment of the present invention, that a resource is an IOPS is used as an example, and generally, one user request consumes one IOPS; or that a resource is network bandwidth is used as an example, and a resource that is consumed by one user request is network bandwidth for the user request. When a resource consumed by a user request is network bandwidth, a size of a resource that is provided by a resource-providing entity and consumed by each user request is determined by the user request.

In this embodiment of the present invention, scheduling a user request by using a scheduling algorithm includes: computing a virtual start time and a virtual finish time of the user request according to the scheduling algorithm in this embodiment of the present invention; and adding the user request to a scheduling queue, where the user request is ranked in the scheduling queue according to a value of the virtual start time of the user request. When it is not required to schedule the user request, the virtual start time and the virtual finish time of the user request need to be computed according to the scheduling algorithm in this embodiment of the present invention, and the user request does not need to be added to a scheduling queue for ranking.

In this embodiment of the present invention, S_(d), S_(x), R_(x), G_(y), and G_(k) are merely used to mark a specific device each, where the device herein may be a physical device or a logical device. Likewise, a device may also be expressed in a manner of first, second, and the like. There is only a difference in specific expression manners, which does not impose a limitation on the solution of the present invention.

As shown in FIG. 12, an embodiment of the present invention provides a scheduler S_(d). The scheduler S_(d) is applied to a distributed resource system, where the distributed resource system includes schedulers S_(x), resource-providing entities R_(x), and a coordinator G_(y), where x is any one of consecutive natural numbers from 1 to M, and M≧2; d is a number in x; y is any one of consecutive natural numbers from 1 to Y; the S_(x) communicates with the R_(x); the G_(y) communicates with any S_(x); and the scheduler S_(d) includes an acquiring unit 1301 and a scheduling unit 1302, where:

the acquiring unit 1301 is configured to acquire, in a T_(n+1) period and from a coordinator G_(k) of a user z, a resource C_(z)(T_(n)) that is consumed by a user z request in a T_(n) period, where

${{C_{z}\left( T_{n} \right)} = {\sum\limits_{x = 1}^{M}{C_{z,x}\left( T_{n} \right)}}};$

k is a natural number, and 1≦k≦Y; a resource weight of the user z is φ_(z); C_(z,x)(T_(n)) is a quantity of resources that are provided by the R_(x) and consumed by N_(z,x)(T_(n)) user z requests received by the S_(x) in the T_(n) period; and z indicates an identifier of the user; and

the scheduling unit 1302 is configured to schedule, according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)), a P^(i) _(z,d) by using a scheduling algorithm, where the P^(i) _(z,d) is the i^(th) user z request received by the S_(d), and C_(z,d)(T_(n)) is a quantity of resources that are provided by an R_(d) and consumed by N_(z,d)(T_(n)) user z requests received by the S_(d) in the T_(n) period.

Specifically, that the scheduling unit 1302 is configured to schedule, according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)), a P^(i) _(z,d) by using a scheduling algorithm specifically includes:

computing a virtual start time S(P^(i) _(z,d)) and a virtual finish time F(P^(i) _(z,d)) of the P^(i) _(z,d) according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)); and adding the P^(i) _(z,d) to a scheduling queue, where the scheduling queue ranks the user request according to a value of the virtual start time of the user request.

Specifically, S(P^(i) _(z,d))=max{v(P^(i) _(z,d)), F(P^(i−1) _(z,d))},

${{F\left( P_{z,d}^{i} \right)} = {{S\left( P_{z,d}^{i} \right)} + \frac{c\left( P_{z,d}^{i} \right)}{\varphi_{z}} + \frac{d\left( {P_{z,d}\left( T_{n + 1} \right)} \right)}{\varphi_{z}}}},{and}$ ${{d\left( {P_{z,d}\left( T_{n + 1} \right)} \right)} = \frac{{C_{z}\left( T_{n} \right)} - {C_{z,d}\left( T_{n} \right)}}{N_{z,d}\left( T_{n} \right)}},$

where v(P^(i) _(z,d)) indicates a virtual time of the S_(d) when the S_(d) receives the P^(i) _(z,d), and c(P^(i) _(z,d)) indicates a resource that is provided by the R_(d) and consumed by the P^(i) _(z,d).

Specifically, S(P^(i) _(z,d))=max{v(P^(i) _(z,d)),

$\left. {{F\left( P_{z,d}^{i - 1} \right)} + \frac{d\left( {P_{z,d}\left( T_{n + 1} \right)} \right)}{\varphi_{z}}} \right\},{{F\left( P_{z,d}^{i} \right)} = {{S\left( P_{z,d}^{i} \right)} + \frac{c\left( P_{z,d}^{i} \right)}{\varphi_{z}}}},{and}$ ${{d\left( {P_{z,d}\left( T_{n + 1} \right)} \right)} = \frac{{C_{z}\left( T_{n} \right)} - {C_{z,d}\left( T_{n} \right)}}{N_{z,d}\left( T_{n} \right)}},$

where v(P^(i) _(z,d)) indicates a virtual time of the S_(d) when the S_(d) receives the P^(i) _(z,d), and c(P^(i) _(z,d)) indicates a resource that is provided by the R_(d) and consumed by the P^(i) _(z,d).

For meanings of the virtual start time S(P^(i) _(A,A)) the virtual finish time F(P^(i) _(A,A)), and the virtual time v(P^(i) _(A,A)), refer to a Start-time Fairness Queuing (SFQ) algorithm, and details are not described in this embodiment of the present invention again.

Specifically, when either of C_(z)(T_(n))-C_(z,d)(T_(n)) and N_(z,d) (T_(n)) is 0, d(P_(z,d)(T_(n+1)))=0.

The S_(d) schedules, according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)) the P^(i) _(z,d) by using the scheduling algorithm. The user z request can be scheduled without depending on a user agent. In addition, the S_(d) schedules, according to φ_(z), C_(z)(T_(n)), C_(z,d)(T_(n)), and N_(z,d)(T_(n)), the P^(i) _(z,d) by using the scheduling algorithm, thereby implementing global scheduling on the user z request and ensuring a performance requirement of the user z.

In an implementation manner of another resource scheduling method in a distributed resource system, the distributed resource system includes multiple schedulers, where a first scheduler in the multiple schedulers acquires, from a coordinator of a first user, the sum of resources that are consumed by a user request of the first user in the multiple schedulers in a previous period; and

the first scheduler schedules the user request of the first user according to a resource weight of the first user, the sum of resources that are consumed by the user request of the first user in the multiple schedulers in the previous period, a resource that is consumed by the user request of the first user in the first scheduler in the previous period, and a quantity of user requests of the first user received by the first scheduler in the previous period.

Further, that the first scheduler schedules the user request of the first user according to a resource weight of the first user, the sum of resources that are consumed by the user request of the first user in the multiple schedulers in the previous period, a resource that is consumed by the user request of the first user in the first scheduler in the previous period, and a quantity of user requests of the first user received by the first scheduler in the previous period specifically includes that:

the first scheduler computes a virtual start time and a virtual finish time of the user request of the first user according to the resource weight of the first user, the sum of resources that are consumed by the user request of the first user in the multiple schedulers in the previous period, the resource that is consumed by the user request of the first user in the first scheduler in the previous period, and the quantity of user requests of the first user received by the first scheduler in the previous period; and adds the user request of the first user to a scheduling queue, where the scheduling queue ranks the user request of the first user according to a value of the virtual start time of the user request.

Another distributed resource system includes multiple schedulers. As shown in FIG. 12, a first scheduler in the multiple schedulers includes an acquiring unit 1301 and a scheduling unit 1302, where:

the first acquiring unit 1301 is configured to acquire, from a coordinator of a first user, the sum of resources that are consumed by a user request of the first user in the multiple schedulers in a previous period; and

the scheduling unit 1302 is configured to schedule the user request of the first user according to a resource weight of the first user, the sum of resources that are consumed by the user request of the first user in the multiple schedulers in the previous period, a resource that is consumed by the user request of the first user in the first scheduler in the previous period, and a quantity of user requests of the first user received by the first scheduler in the previous period.

Further, the scheduling unit 1302 is specifically configured to compute a virtual start time and a virtual finish time of the user request of the first user according to the resource weight of the first user, the sum of resources that are consumed by the user request of the first user in the multiple schedulers in the previous period, the resource that is consumed by the user request of the first user in the first scheduler in the previous period, and the quantity of user requests of the first user received by the first scheduler in the previous period; and add the user request of the first user to a scheduling queue, where the scheduling queue ranks the user request of the first user according to a value of the virtual start time of the user request.

The technical solutions provided by the embodiments of the present invention may also be applied to another scenario. For example, a distributed resource system is a distributed computing system, and a resource-providing entity provides a computing resource for a user request; or, a distributed resource system may be a distributed network system, and a resource-providing entity provides network bandwidth for a user request; or, a resource-providing entity may further provide a memory resource for a user request.

A person of ordinary skill in the art may be aware that, in combination with the examples described in the embodiments disclosed in this specification, units and algorithm steps may be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed by hardware or software depends on particular applications and design constraint conditions of the technical solutions.

It may be clearly understood by a person skilled in the art that, for the purpose of convenient and brief description, for a detailed working process of the foregoing system, apparatus, and unit, refer to a corresponding process in the foregoing method embodiments, and details are not described herein again.

In the several embodiments provided in the present application, it should be understood that the unit division is merely logical function division and may be other division in actual implementation. For example, a plurality of units or components may be combined or integrated into another system, or some features may be ignored or not performed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented through some interfaces. The indirect couplings or communication connections between the apparatuses or units may be implemented in electronic, mechanical, or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each of the units may exist alone physically, or two or more units are integrated into one unit.

When the functions are implemented in the form of a software functional unit and sold or used as an independent product, the functions may be stored in a computer-readable non-volatile storage medium. Based on such an understanding, the technical solutions of the present invention essentially, or the part contributing to the prior art, or some of the technical solutions may be implemented in a form of a software product. The software product is stored in a non-volatile storage medium, and includes several instructions for instructing a computer device (which may be a personal computer, a server, or a network device) to perform all or some of the steps of the methods described in the embodiments of the present invention. The foregoing non-volatile storage medium includes: any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (Read-Only Memory, ROM), a magnetic disk, or an optical disc. 

What is claimed is:
 1. A resource scheduling method in a distributed resource system, wherein the distributed resource system comprises multiple schedulers; a first scheduler in the multiple schedulers acquiring, from a coordinator of afirst user, asum of resources that are consumed in the multiple schedulers by a user request of the first user in a previous period; and the first scheduler scheduling the user request of the first user according to a resource weight of the first user, the sum of resources that are consumed in the multiple schedulers by the user request of the first user in the previous period, a resource that is consumed in the first scheduler by the user request of the first user in the previous period, and a quantity of user requests of the first user received by the first scheduler in the previous period.
 2. The method according to claim 1, wherein that the first scheduler schedules the user request of the first user according to the resource weight of the first user, the sum of resources that are consumed in the multiple schedulers by the user request of the first user in the previous period, a resource that is consumed in the first scheduler by the user request of the first user in the previous period, and the quantity of user requests of the first user received by the first scheduler in the previous period comprises: computing, by the first scheduler according to the resource weight of the first user, the sum of resources that are consumed in the multiple schedulers by the user request of the first user in the previous period, the resource that is consumed in the first scheduler by the user request of the first user in the previous period, and the quantity of the user requests of the first user received by the first scheduler in the previous period; computing a virtual start time and a virtual finish time of the user request of the first user; and adding the user request of the first user to a scheduling queue; wherein the scheduling queue ranks the user request of the first user according to a value of the virtual start time of the user request.
 3. A first scheduler in a distributed resource system comprising multiple schedulers, the first scheduler comprising a central processing unit and a memory, wherein the central processing unit (CPU) executes an executable instruction in the memory to perform the following steps: acquiring, from a coordinator of a first user, a sum of resources that are consumed in the multiple schedulers by a user request of the first user in a previous period; and scheduling the user request of the first user according to a resource weight of the first user, the sum of resources that are consumed in the multiple schedulers by the user request of the first user in the previous period, a resource that is consumed in the first scheduler by the user request of the first user in the previous period, and a quantity of user requests of the first user received by the first scheduler in the previous period.
 4. The scheduler according to claim 3, wherein the CPU further performs: computing, according to the resource weight of the first user, the sum of resources that are consumed in the multiple schedulers by the user request of the first user in the previous period, the resource that is consumed in the first scheduler by the user request of the first user in the previous period, and the quantity of the user requests of the first user received by the first scheduler in the previous period; computing a virtual start time and a virtual finish time of the user request of the first user; and adding the user request of the first user to a scheduling queue; wherein the scheduling queue ranks the user request of the first user according to a value of the virtual start time of the user request.
 5. A distributed resource system, wherein the distributed resource system comprises multiple schedulers, a coordinator for a first user is configured to provide toa first scheduler in the multiple schedulers, a sum of resources that are consumed in the multiple schedulers by a user request of the first user in a previous period; the first scheduler is configured to: acquire, from the coordinator of the first user, the sum of resources that are consumed in the multiple schedulers by the user request of the first user in the previous period; and schedule the user request of the first user according to a resource weight of the first user, the sum of resources that are consumed in the multiple schedulers by the user request of the first user in the previous period, a resource that is consumed in the first scheduler by the user request of the first user in the previous period, and a quantity of user requests of the first user received by the first scheduler in the previous period.
 6. The distributed resource system according to claim 5, wherein the first scheduler is further configured to: compute, according to the resource weight of the first user, the sum of resources that are consumed in the multiple schedulers by the user request of the first user in the previous period, the resource that is consumed in the first scheduler by the user request of the first user in the previous period, and the quantity of the user requests of the first user received by the first scheduler in the previous period; compute a virtual start time and a virtual finish time of the user request of the first user; and add the user request of the first user to a scheduling queue; wherein the scheduling queue ranks the user request of the first user according to a value of the virtual start time of the user request. 